GPU-Based One-Dimensional Convolution for Real-Time Spatial Sound Generation

Incorporating spatialized (3D) sound cues in dynamic and interactive videogames and immersive virtual environment applications is beneficial for a number of reasons, ultimately leading to an increase in presence and immersion. Despite the benefits of spatial sound cues, they are often overlooked in videogames and virtual environments where typically, emphasis is placed on the visual cues. Fundamental to the generation of spatial sound is the one-dimensional convolution operation which is computationally expensive, not lending itself to such real-time, dynamic applications. Driven by the gaming industry and the great emphasis placed on the visual sense, consumer computer graphics hardware, and the graphics processing unit (GPU) in particular, has greatly advanced in recent years, even outperforming the computational capacity of CPUs. This has allowed for real-time, interactive realistic graphics-based applications on typical consumer- level PCs. Given the widespread use and availability of computer graphics hardware and the similarities that exist between the fields of spatial audio and image synthesis, here we describe the development of a GPU-based, one-dimensional convolution algorithm whose efficiency is superior to the conventional CPU-based convolution method. The primary purpose of the developed GPU-based convolution method is the computationally efficient generation of real- time spatial audio for dynamic and interactive videogames and virtual environments

User Assessment in Serious Games and Technology-Enhanced Learning

1 Department of Naval, Electric, Electronic and Telecommunications Engineering, University of Genoa, Via all'Opera Pia 11/a, 16145 Genoa, Italy 2 Faculty of Business and Information Technology, University of Ontario Institute of Technology, 2000 Simcoe Street North, Oshawa, Canada L1H 7K4 3Department of Mechanical and Aerospace Engineering, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH 44106, USA 4 Faculty of Computer Science, Universidad Complutense de Madrid, Ciudad Universitaria Universidad Complutense de Madrid, 28040 Madrid, Spai

Assessment in and of serious games: an overview

There is a consensus that serious games have a significant potential as a tool for instruction. However, their effectiveness in terms of learning outcomes is still understudied mainly due to the complexity involved in assessing intangible measures. A systematic approach—based on established principles and guidelines—is necessary to enhance the design of serious games, and many studies lack a rigorous assessment. An important aspect in the evaluation of serious games, like other educational tools, is user performance assessment. This is an important area of exploration because serious games are intended to evaluate the learning progress as well as the outcomes. This also emphasizes the importance of providing appropriate feedback to the player. Moreover, performance assessment enables adaptivity and personalization to meet individual needs in various aspects, such as learning styles, information provision rates, feedback, and so forth. This paper first reviews related literature regarding the educational effectiveness of serious games. It then discusses how to assess the learning impact of serious games and methods for competence and skill assessment. Finally, it suggests two major directions for future research: characterization of the player's activity and better integration of assessment in games

Specialized CNT-based Sensor Framework for Advanced Motion Tracking

In this work, we discuss the design and development of an advanced framework for high-fidelity finger motion tracking based on Specialized Carbon Nanotube (CNT) stretchable sensors developed at our research facilities. Earlier versions of the CNT sensors have been employed in the high-fidelity finger motion tracking Data Glove commercialized by Yamaha, Japan. The framework presented in this paper encompasses our continuing research and development of more advanced CNT-based sensors and the implementation of novel high-fidelity motion tracking products based on them. The CNT sensor production and communication framework components are considered in detail and wireless motion tracking experiments with the developed hardware and software components integrated with the Yamaha Data Glove are reported

Communicating with Humans and Robots: A Motion Tracking Data Glove for Enhanced Support of Deafblind

In this work, we discuss the design and development of a communication system for enhanced support of the deafblind. The system is based on an advanced motion tracking Data Glove that allows for high fidelity determination of finger postures with consequent identification of the basic Malossi alphabet signs. A natural, easy-to-master alphabet extension that supports single-hand signing without touch surface sensing is described, and different scenarios for its use are discussed. The focus is on using the extended Malossi alphabet as a communication medium in a Data Glove-based interface for remote messaging and interactive control of mobile robots. This may be of particular interest to the deafblind community, where distant communications and robotized support and services are rising. The designed Data Glove-based communication interface requires minimal adjustments to the Malossi alphabet and can be mastered after a short training period. The natural interaction style supported by the Data Glove and the popularity of the Malossi alphabet among the deafblind should greatly facilitate the wider adoption of the developed interface

THE SONEL MAPPING ACOUSTICAL MODELING METHOD

This work investigates the application of photon mapping to model environmental acoustics. The resulting acoustic sonel mapping technique is a Monte-Carlo based approach that can be used to model acoustic environments while accounting for diffuse and specular acoustic reflections as well as diffraction effects. This modeling is performed in an efficient manner in contrast to available deterministic techniques. The sonel mapping approach models many of the subtle interaction effects required for realistic acoustical modeling. iv Acknowledgments Wow, the dissertation is complete! Now comes the difficult task of remembering and thanking all those that have helped me throughout my Ph.D journey. First and foremost, I would like to thank my supervisor Professor Michael Jenkin for all his help, support and very, very generous financial assistance throughout my Ph.D (and M.Sc) studies at York University. Despite his very busy schedule, Professor Jenkin always made time for any questions I may have had regarding my work and always had an answer! His door was always open and his email responses were amazingly prompt! I also very much appreciate all the advice and guidance that he provided me with during my search for a faculty position as well as his advice and guidance after obtaining a faculty position. I would like to also thank my co-supervisor Professor Evangelos Milios for all his help, support and guidance throughout all of my academic career. Professor Milios actually provided me with my first research opportunity during my first undergraduate year (an in-depth search of “how animals climb”) and opened the doors to a “whole new research world ” for me. As they say, the rest is history! A thank you also goes out to the other members of my dissertation committee who were very accommodating in setting up a defense during the vacation month of July and for their comments and feedback: Professo

Decruitment of the Perception of Changing Sound Intensity for Simulated Self-Motion

Of the many cues that could be used to gauge selfmotion, auditory cues seem to be the least studied. Listeners could potentially use either a sweep of rising sound intensity to judge their self-motion towards an object or conversely use a sweep of falling sound intensity to judge their motion away from an object. Whether the sweep is rising or falling the listener must judge both the change in intensity across the sweep, and the temporal span of the sweep. Studies indicate that sweeping intensities are misperceived so that the sound intensity at the end of the sweep is judged differently than when the final sound intensity is presented alone. Although there is ongoing discussion as to whether the induced fading is greater for rising sound intensity as opposed to falling sound intensity, both phenomena affect the perception of self-motion. This paper presents a series of experiments that examined self-motion perception with auditory cues. Results confirm the finding of decruitment for a sweeping broadband sound source that decreases at various rates of acceleration. Furthermore, the phenomenon of decruitment was greatly diminished at higher accelerations indicating that this phenomenon is likely correlated to the lowest rate at which listeners can perceive a change in intensity